A computer in operation includes hardware, software, and data. The hardware typically includes a processor, memory, storage, and I/O (input/output) devices coupled together by a bus. The software typically includes an operating system and applications. The applications perform useful work on the data for a user or users. The operating system provides an interface between the applications and the hardware. The operating system performs two primary functions. First, it allocates resources to the applications. The resources include hardware resources—such as processor time, memory space, and I/O devices—and software resources including some software resources that enable the hardware resources to perform tasks. Second, it controls execution of the applications to ensure proper operation of the computer.
Often, the software is conceptually divided into a user level, where the applications reside and which the users access, and a kernel level, where the operating system resides and which is accessed by system calls. Within an operating computer, a unit of work is referred to as a process. A process is computer code and data in execution. The process may be actually executing or it may be ready to execute or it may be waiting for an event to occur. System calls provide an interface between the processes and the operating system.
Checkpointing is a technique employed on some computers where an intermediate computational state of processes is captured. When processes take significant time to execute, they are susceptible to intervening system failures. By occasionally performing a checkpoint of processes and resources assigned to processes, the processes can be restarted at an intermediate computational state in an event of a system failure. Migration is a technique in which running processes are checkpointed and then restarted on another computer. Migration allows some processes on a heavily used computer to be moved to a lightly used computer. Checkpointing, restart, and migration have been implemented in a number of ways.
Operating system checkpoint, restart, and migration have been implemented as an integral part of several research operating systems. However, such research operating systems are undesirable because they lack an installed base and, consequently, few applications exist for them. Application level checkpoint, restart, and migration in conjunction with standard operating systems have also been implemented. But these techniques require that processes not use some common operating system services because the checkpointing only takes place at the application level.
Object based checkpoint, restart, and migration have also been implemented. Such object based approaches use particular programming languages or middleware toolkits. The object based approaches require that the applications be written in one of the particular programming languages or that the applications make explicit use of one of the middleware toolkits. A virtual machine monitor approach can be used to implement checkpoint, restart, and migration. But such an approach requires checkpointing and restarting all processes within the virtual machine monitor. This approach also exhibits poor performance due to isolation of the virtual machine monitor from an underlying operating system.
In The Design and Implementation of Zap: A System for Migrating Computing Enviroments, Proc. OSDI 2002, Osman et al. teach a technique of adding a loadable kernel module to a standard operating system to provide checkpoint, restart, and migration of processes for existing applications. The loadable kernel model divides the user level into process domains and provides virtualization of resources within each process domain. Such virtualization of resources includes virtual process identifiers and virtualized network addresses. Processes within one process domain are prevented from interacting with processes in another process domain using inter-process communication techniques. Instead, processes within different process domains interact using network communications and shared files set up for communication between different computers.
Checkpointing in the technique taught by Osman et al. records the processes in a process domain as well as the state of the resources used by the processes. Because resources in the process domain are virtualized, restart or migration of a process domain includes restoring resource identifications to a virtualized identity that the resources had at the most recent checkpoint.
While the checkpoint, restart, and migration techniques taught by Osman et al. show promise, several areas could be improved. In particular, these techniques are only capable of working with processes within a single process domain. There is no technique for checkpointing parallel processes executing within a plurality of process domains.
What is needed is a method of checkpointing parallel processes operating within a plurality of process domains.